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Abstract 

We study, both numericahy and analytically, a Binary- Agent-Resource (B-A-R) model consisting 
of N agents who compete for a limited resource 1/2 < L/N < 1, where L is the maximum available 
resource per turn for all N agents. As L increases, the system exhibits well-defined plateaux regions 
in the success rate which are separated from each other by abrupt transitions. Both the maximum 
and the mean success rates over each plateau are 'quantized' - for example, the maximum success 
rate forms a well-defined sequence of simple fractions as L increases. We present an analytic theory 
which explains these surprising phenomena both qualitatively and quantitatively. The underlying 
cause of this complex behavior is an interesting self-organized phenomenon in which the system, in 
response to the global resource level, effectively avoids particular patterns of historical outcomes. 
PACS Nos.: 02.50.Le, 05.65.-hb, 05.40.-a, 89.90.+n 



I. INTRODUCTION 



Complex systems have attracted much attention among physicists, apphed mathemati- 
cians, engineers, and social scientists in recent years. In particular, agent-based models have 
become an important part of research on Complex Adaptive Systems For example, self- 
organized phenomena in an evolving population consisting of agents competing for a limited 
resource, have potential applications in areas such as engineering, economics, biology, and 
social sciences . The bar-attendance problem proposed by Arthur [3|, \^ constitutes an 
everyday example of such a system in which a population of agents decide whether to go to 
a popular bar having limited seating capacity. The agents are informed of the attendance in 
past weeks, and hence share common information, make decisions based on past experience, 
interact through their actions, and in turn generate this common information collectively. 
These ingredients are key characteristics of complex systems j^. An important step in the 
recent explosion of research in agent-based models within the physics community, has been 
the introduction of binary Ising-like versions of models of competing populations - exam- 

and the Binary- Agent-Resource (B-A-R) game 



a'es include the Minority Game (MG) 0, 0] 



studies 



. features of such a competing population. It has 
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For modest resource levels in which there are more losers than winners, the Minority 
Game represents a simple, yet highly non-trivial, model that captures many of the 

essential features of such a competing population. It ha s been the subject of many theoretical 

The MG considers an odd number 
of agents. At each timestep, the agents independently decide between two options '0' 
and '1'. The winners are those who choose the minority option. The agents learn from past 
experience by evaluating the performance of their strategies, where each strategy maps the 
available global information, i.e. the record of the most recent m winning options, to an 
action. One important quantity in the MG is the standard deviation a of the number of 
agents making a particular choice. This quantity reflects the performance of the population 
as a whole in that a small a implies on average more winners per turn, and hence a higher 
success rate per turn among the agents. In the MG, a exhibits a non-monotonic dependence 



11, 12, nn. When m is small, there is a significant 
overlap between the agents' strategies. This crowd effect leads to a large a, 



on the memory size m of the agents 



implying the number of losers is high. This is the crowded, or informationally efficient, phase 
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of MG. In the informationally inefficient phase where m is large, a is moderately small and 
the agents perform better than if they were to decide their actions randomly. In this regime, 
information is left in the resulting bit-string patterns for a single realization of the system. 
In the inefficient phase, the MG can be mapped on to disordered spin systems and hence 



the machinery in statistica. 
can be apphed 



physics of disordered systems, most noticeably the replica trick, 
isi [19^ . However, the replica trick becomes ineffective in the 
efficient phase. The Crowd-Anticrowd theory 

mm 

gives a physically transparent, 

quantitative theory of the observed features of the MG in both the efficient and inefficient 
regimes. The Crowd-Anticrowd theory is based on the fact that it is the difference in the 
numbers of agents playing a given strategy R and the corresponding anti-correlated strategy 
i?, that dictates the size of the fluctuations and hence performance of the population as a 
whole. 

In agent-based models, the non-triviality of the results comes from the actions taken by 
the agents which are directly related to the decision mechanism. The decision mechanism 
depends sensitively on how each strategy performs at the moment of decision. In the efficient 
phase of the MG, no strategy outperforms the others and therefore the relative performance 
of the strategies oscillates as the game proceeds. This anti-persistent nature of the strategy 

performance HQ, HQS, 

I24I I25I I is crucial m arriving at a quantitative understanding 



of the MG's dynamics. As the systems evolves, it goes from one m-bit outcome or history 
bit-string to another. Mathematically, the evolution can be viewed in terms of transitions 
in the global information (i.e. history) space. The 2"^ possible history bit-strings for a 
given value of m constitute the nodes in this history space. As the system evolves, it makes 
transitions from one node to another. Jefferies et al. showed that in the efficient (i.e. low 
m) phase of the MG, an effective restoring force dominates the strategy-score dynamics 
yielding a Eulerian Trail quasi-attractor in history space j2j|. As m increases, a competing 
bias term - associated with the initial strategy allocation - becomes increasingly important 
and eventually leads to instability of the Eulerian Trail quasi-attractor [2^ . 

Johnson et al. ^ subsequently introduced and studied numerically what is known as 
the Binary-Agent-Resource (B-A-R) model, in which the winning group is not necessarily 
decided by the minority rule. The B-A-R model features a cutoff parameter L, which is 
referred to as the global resource level L {L < N). The values of L and are not known to 
the agents. At each timestep t, each agent decides upon two possible options: whether to 



3 



access the resource or not. The winning action is decided by whether the number of agents 
attempting to access the resource, actually exceeds this resource level. The MG therefore 
corresponds to the particular case of L = N/2 in the B-A-R model. In Ref.j^, it was 
found numerically that the population may unwittingly self-segregate itself into groups if L 
deviates sufficiently from N/2. As an example, consider a very high resource level oi L N 
with each agent holding two strategies: it was found numerically that approximately 3A^/4 
agents are persistent winners while the rest are persistent losers. 

In the present work we analyze, both numerically and analytically, the transition from 
modest to high resources in this generic B-A-R model of a competing multi-agent popu- 
lation. Surprisingly, we find that the system exhibits a set of abrupt transitions between 
distinct yet well-defined states as the resource level L increases from N/2 to A^. In particular, 
both the highest success rate Wmax and the mean success rate {w) among the agents, show 
abrupt transitions as L varies. In addition, Wmax exhibits fractional values in the plateau 
regions between each transition. We show that this behavior can be understood in terms of 
the system's trajectory in the history space as time evolves. In particular, as L increases, 
the portion of the history space that the system visits becomes increasingly restricted. We 
derive analytic expressions for the observed plateaux values and the values of L at which the 
transitions occur. Although the present analysis focuses on a non-networked population, the 
same elements of (i) strategy performance over time, and hence the dynamics of strategy 
scores, plus (ii) the system's trajectory in history space, provide the foundation for a quanti- 
ative understanding of a large class of agent-based models, including networked populations 
25! l2fi| . We also note that for physicists interested in random walks, the present B-A-R 
system provides a fascinating laboratory for studying correlated, non-Markovian diffusion 
on a non-trivial network (i.e. the history space, which corresponds to a de Bruijn graph). 
This non-trivial diffusion is in turn strongly coupled to the non-random temporal patterns 
arising in the strategy-performance dynamics. Finally, we note in passing that the occur- 
rence of abrupt transitions between plateau states, and stable fractions, are known to arise 
in a multi-electron quantum system as the external magnetic field is increased monotonically 
(i.e. Fractional Quantum Hall Effect [27]). However it is very curious to see such 'quan- 
tized' phenomena arise in a classical multi-particle system as a function of a monotonically 
increasing external control parameter. 

The plan of the paper is as follows. In Sec. II, we define the B-A-R model. In Sec. HI, 
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we present numerical results from extensive simulations. In particular, we demonstrate the 
existence of different phases or states at different values of the resource level. In Sec. IV, we 
discuss how the strategies' performance evolves as the system evolves. In Sec. V, we discuss 
the history space of the B-A-R model and present results for the statistics in the outcome 
bit-strings at high resource levels. In Sec. VI, we explain the observed numerical features 
based on the idea that at high resource levels, the system restricts itself to only visit a 
restricted portion of the history space. We derive an expression for the highest success rate 
Wmax among the agents and discuss the critical values of resource level at which transitions 
occur. We summarize the results in Sec. VII, together with a discussion of how the present 
approach can be generalized to a wider class of agent-based models. 



II. MODEL 

We consider the Binary- Agent-Resource (B-A-R) model 7, 8,0]. The B-A-R model is a 
binary version of Arthur's El Parol bar attendance model 21,1^, in which a population of 
agents repeatedly decide whether to go to a bar with limited seating based on the information 
of the crowd size in recent weeks. In the B-A-R model, there is a global resource level L 
which is not announced to the agents, where is the total number of agents. At each 
timestep t, each agent decides upon two possible options: whether to access resource L 
(action '1') or not (action '0'). The two global outcomes at each timestep, 'resource over- 
used' and 'resource not over-used', are denoted by '0' and '1'. If the number of agents ni(t) 
choosing action 1 exceeds L (i.e. resource over-used and hence global outcome '0') then the 

— ni(t) abstaining agents win. By contrast if ni{t) < L (i.e. resource not over-used and 
hence global outcome '1') then the ni{t) agents win. In order to investigate the behavior 
of the system as L changes, it is sufficient to study the range N/2 < L < N. The results 
for the range < L < N/2 can be obtained from those in the present work by suitably 
interchanging the role of '0' and '1' j^. In the special case of L = N/2, the B-A-R model 
reduces to the Minority Game. 

In the B-A-R model, each agent shares a common knowledge of the past history of the 
most recent m outcomes, i.e. the winning option in the most recent m timesteps. The full 
strategy space thus consists of 2^™ strategies, as in the MG. Initially, each agent randomly 
picks s strategies from the pool of strategies, with repetitions allowed. The agents use these 
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strategies throughout the game. At each timestep, each agent uses his momentarily best 
performing strategy with the highest virtual points. The virtual points for each strategy 
indicate the cumulative performance of that strategy: at each timestep, one virtual point 
(VP) is awarded (deducted) to (from) a strategy that would have predicted the correct 
(incorrect) outcome after all decisions have been made. A random coin-toss is used to 
break ties between strategies. In the B-A-R model, the population may or rnay not contain 
network connections. In the case 

has access to additional information from his connected neighbors, such as his neighbors' 
strategies and/or performance. In the present work, we focus on the B-A-R model with a 
non-networked population. 

To evaluate the performance of an agent, one (real) point is awarded to each winning 
agent at a given timestep. A maximum of L points per turn can therefore be awarded to 
the agents. An agent has a success rate w, which is the mean number of points awarded to 
the agent per turn over a long time window. The mean success rate {w) among the agents 
is then defined to be the mean number of points awarded per agent per turn, i.e. an average 
of w over the agents. We are interested in investigating the details of how the success rate 
changes as the resource level L varies in the efficient phase, where the number of strategies 
(repetitions counted) in play is larger than the total number of distinct strategies in the 
strategy space. 



III. NUMERICAL RESULTS: RESOURCE-DRIVEN STATES IN B-A-R MODEL 

n 

The effects of varying L were ffist reported by Johnson et al. [9]. These authors studied 
numerically the dependence of the fluctuations in the number of agents taking a particular 
option, on the memory size m for different values of L. For the MG (i.e. L = N/2) in 
the efficient phase (i.e. small values of m) the number of agents making a particular choice 
varies from timestep to timestep, with additional stochasticity introduced via the random 
tie-breaking process. The corresponding period depends on the memory length m. The 
underlying reason is that in the efficient phase for L = N/2, no strategy is better overall 
than any other. Hence there is a tendency for the system to restore itself after a finite 
number of timesteps, thereby preventing a given strategy's VPs from running away from the 
others. As a result, the outcome bit-string shows the feature of anti-persistency or double 
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periodicity ,|liQ,Q,|2liy,|24|. Since a maximum of L = points can be awarded 
per turn, the mean success rate {w) over a sufficiently large number of timesteps is bound 
from above hj L/N = 1/2. 

In the B-A-R system with high resource level, the mean success rate behaves differently. 
Taking the extreme case of L ~ A^, the winning action is obviously '1' (i.e. access resource) 
and in principle every agent could win in every timestep. The history upon which the agents 
decide, is persistently m-bits off. However, due to the random initial strategy distribution, 
some agents may not hold a strategy that predicts the winning option for a history of m 
'f s. Therefore, there are still losers and (w) is less than L. The number of losers depends 
on s, the number of strategies that each agent holds. For s = 2 and assuming that the 
strategies are picked randomly, a mean number of N/ 4 agents in a large population will hold 
two strategies both predicting the wrong option. The mean success rate is thus given by 
(w) = 3/4. 

We have carried out extensive numerical simulations on the B-A-R model to investigate 
the dependence of the success rate on L for N/2 < L < N. Unless stated otherwise, we 
consider systems with A^ = 1001 agents and s = 2. Figure^a) shows the results of the mean 
success rate (dark solid line) as a function of L in a typical run for m = 3, together with the 
range corresponding to one standard deviation about (w) in the success rates among the A^ 
agents (dotted lines) and the spread in the success rates given by the highest and the lowest 
success rates (thin solid lines) in the population. By taking a larger value of A^ than most 
studies in the literature, we can analyze the dependence on L and m in great detail. In 
particular, these quantities all exhibit abrupt transitions (i.e. jumps) at particular values of 
L. Between the jumps, the quantities remain essentially constant and hence form steps or 
'plateaux'. We refer to these different plateaux as states or phases, since it turns out that 
the jump occurs when the system makes a transition from one type of state characterizing 
the outcome bit-string to another. For different runs, the results are almost identical. At 
most, there are tiny shifts in the L values at which jumps arise due to (i) different initial 
strategy distributions among the agents in different runs, and (ii) different random initial 
history bit-strings used to start the runs. 

These different states are most clearly seen by monitoring the highest success rate Wmax 
among the agents for given values of L and m. The most striking feature in Fig^a) is that 
the values of the plateaux in Wmax are given by simple fractions, e.g. 7/8, 6/7, 5/6, 12/17, 
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1/2, etc. This feature strongly suggests that the system goes through different states with 
different ratios of '1' and '0' in the outcome bit-string as L varies, as will be discussed in 
later sections. Figure Q^b) shows that the features in the success rates for the simpler case 
of m = 1 are similar to those in Fig^a), except that the plateaux in Wmax take on fewer 
values, i.e. 1, 3/4, 1/2 as L decreases. These values are closely related to the statistics in 
the outcome bit-string. For large and m = 1, the outcome bit-string shows a period of 
4 bits. For values of L with Wmax = 1/2, it turns out that the fraction of the outcome '1' 
in a period is exactly 1/2. For the range of L corresponding to Wmax = 3/4, there are three 
I's in a period of 4, and so on. For m = 3, we have also carried out detailed analysis of the 
outcome bit-string. For later discussions, we summarize in Table E] the values of Wmax, the 
range of L corresponding to the observed value of Wmax, the ratio of number of occurrence 
of 'I'-bits to 'O'-bits and the period in the outcome bit-string, as obtained numerically from 
the data shown in Fig|T] Hereafter, Wmax is used to label the state at a given L. 

IV. AGENTS' DECISIONS AND STRATEGY PERFORMANCE 

The agents decide based on the best performing strategy that they hold at the moment 
of their decision. A strategy's performance is evaluated by its virtual points, which vary 
as the game proceeds. It is most illustrative to consider the case of m = 1 and s = 2 (see 
Fig^b)) since one can readily follow the dynamics for different values of L. For m = 1, 
there are only four different strategies in the whole strategy space. These strategies can be 
represented by (00), (01), (10), and (11), with the first (second) index in (xy) giving the 
action for the history bit-string of '0' and '1', respectively. The virtual points (VP) of the 
four strategies can then be represented by a 2 x 2 matrix VP^.^, with the xy-element giving 
the VP of the strategy (xy). Table HTl shows the time evolution of the VPs of the strategies in 
a few timesteps, the number of agents rii taking the action '1', and the outcome for m = 1 in 
a population with uniform initial distribution of all possible pairs of strategies to the agents. 
We illustrate these ideas by considering the case of an initial history of '0' for given L, but 
the same results are obtained for an initial history of '1'. For N/2 < L < llA^/16, the system 
follows the dynamics shown in the left column. In this range of L, the VPs cannot run away 
due to the decision making process of the agents. The VPs restore their values in a few 
timesteps, as in the MG. For systems of large A^, the outcome series shows a 4-bit periodic 
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pattern of 1100, with Wmax = 1/2 in agreement with the numerical results in Fig.^b). This 
highest success rate is achieved by the agents who hold two identical strategies. The result 
also implies that an agent's success rate is determined by the Hamming distance between 
the two strategies that the agent holds js^l- One could therefore say that different 'species' 
of agent emerge in the population due to the dynamics of the system, with each species 
characterized by its own Hamming distance 'gene'. 

For llA^/16 < L < 3N/4, two of the four strategies will have runaway VPs, with one 
tending to increase without bound while another tends to decrease without bound (see 
central column in Table |H|. Following the dynamics, the outcome series shows a 4-bit 
periodic pattern of 1110, in the limit of large A^. In this range, Wmax = 3/4 as observed 
numerically and this value is achieved by those agents holding the strategy whose VPs 
increase without bound. For L > 3N/A, two strategies have their VPs increasing (decreasing) 
without bound (see right column in Table ITT]) . The outcome series is persistently '1', i.e. 
it becomes effectively period-1. In this high resource level regime, (w) =3/4 since three- 
quarters of the agents hold at least one strategy which predicts the persistently winning 
option. In this way, it is possible to follow the dynamics of the system and obtain the 
number of possible states and the range of L for each state. Table IIIII summarizes the 
theoretical results for m = 1 and s = 2, by following the analysis on the dynamics as shown 
in Table |H] The results for the location of the transitions, the values of (w) and Wmax are 
all in good agreement with numerical data (see Fig. ^b)). The important point is that for 
small m, the system undergoes several changes of state with successively higher values of 
(w) and Wmax as L increases. The value of Wmax is related to the ratio of occurrences of 
the two possible outcomes in the outcome series, which in turn is related to the strategies' 
performance and thus the decision-making process for a given value of L. 

We have carried out similar analysis for the case of m = 2 and m = 3. For m = 3, 
the fraction of '1' in a period is found to take on values in the set {^,Y|,^,|,f,|,l}, 
which coincides with the numerical values of Wmax obtained by numerical simulations (see 
Fig. ^a)) and shown in Tabled However, the analysis becomes increasingly complicated 
for higher values of m and/or s. The reason is that there are 2^™ strategies, and allowing s 
strategies per agent divides the agents into (2^™)** groups according to the sequence in which 
an agent picks his s strategies. This number increases rapidly with m and s, and the above 
microscopic analysis becomes hard to implement. It turns out the states are closely related 
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to the way in which the system explores the possible histories. In what follows, we analyze 
the B-A-R model by an approach that focuses on the transitions between history bit-strings 
and hence on the path in the history space j^, • 

V. HISTORY SPACE AND BIT-STRING STATISTICS 
A. History space 



Our approach couples together consideration of the probability o: 



the occurrence of var- 



ious histories and the ranking in the performance of the strategies 2^. The history space 
consists of all the possible history bit-strings for a given value of m. For m = 3, it includes 
2^ bit-strings of O's and I's. Figure Efa) shows the history space for m = 3, together with 
the possible transitions from one history to another. Each history constitutes a node in the 
history space. The transitions are marked by arrows together with the outcome necessary 
for making the transitions. It will prove convenient to group the possible history bit-strings 
for a given m into columns, in the way shown in Fig. Efa). Each column is labelled by a 
parameter (, which is the number of 'O's in the 3-bit history (histories) concerned. One 
immediate advantage of this labelling scheme is that the different states characterized by 
Wmax turn out to involve paths in a restricted portion of the full history space. For example, 
the state with Wmax = 1 is restricted to the C = portion of the history space, i.e. the 111 
history bit-string leads to an outcome of '1' and hence persistent self-looping at the node 111 
in history space. The states with Wmax = 7/8, 6/7, and 5/6 correspond to different paths in 
the history space restricted to the C = ^"^^ C = 1 groups of histories, as shown in Fig. |2fb). 
The states with Wmax = 17/23 and 12/17 have paths extended to include C = 2 histories. 
The state with Wmax = 1/2 has paths that cover the whole history space {( = 0, 1, 2, 3). In 
general, the deviation of L from N/2 acts like a driving force in the history space that drifts 
the system towards an increasingly restrictive portion of the history space bounded by a 
smaller value of (. One can also view this behavior as the system, in response to the global 
resource level L, effectively avoiding certain nodes in the history space and hence avoiding 
particular patterns of historical outcomes. 
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B. Bit-string statistics of different states 



As the game proceeds, the system evolves from one history bit-string to another. This 
can be regarded as transitions between different nodes (i.e. different histories) in the history 
space. For L = N/2 in the efficient phase, it has been shown that the conditional 
probability of an outcome of, say, '1' following a given history is the same for all histories. 
For L 7^ N/2, the result still holds for states characterized by Wmax = 1/2. Note that a 
history bit-string can only make transitions to history bit-strings that differ by the most 
recent outcome, e.g. Ill can only be make transitions to either 110 or 111, and thus many 
transitions between two nodes in the history space are forbidden. In addition, these allowed 
transitions do not in general occur with equal probabilities. This leads to specific outcome 
(and history) bit-string statistics for a state characterized by Wmax- 

We have carried out detailed analysis of the statistics of the outcomes following a given 
history bit-string for m = 3, and for each of the possible states over the whole range of 
L. Table IIVI gives the relative numbers of occurrences of each outcome for every history 
bit-string. For the state with Wmax = 1/2 = 8/16, for example, the outcomes '0' and 
'1' occur with equal probability for every history bit-string, as in the MG. For the other 
states, the results reveal several striking features. It turns out that Wmax is given by the 
relative frequency of an outcome of '1' in the outcome bit-string, which in turn is governed 
by the resource level L. For example, a '0' to '1' ratio of 5 : 12 in the outcome bit-strings 
corresponds to the state with Wmax = 12/17. In Table HV] we have intentionally grouped 
the history bit-strings into rows according to the label ( in Fig. |21 We immediately notice 
that for every possible state in the B-A-R model, the relative frequency of each outcome 
is a property of the group of histories having the same label ( rather than the individual 
history bit-string, i.e. all histories in a group have the same relative fraction of a given 
outcome. This observation is important in understanding the dynamics in the history space 
for different states in that it is no longer necessary to consider each of the 2™ history bit- 
strings in the history space. Instead, it is sufficient to consider the four groups of histories 
(for m = 3) as shown in Fig. |21^a). Analysis of results for higher values of m show the same 
feature. 

For the state characterized by Wmax = 1? the outcome bit-string is persistently '1' and 
the path in the history space is repeatedly 111— >1. Therefore, the path is restricted to the 
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history labelled by ( = and simply corresponds to an infinite number of loops around the 
history node 111. Since there is no '0' in the outcome bit-string, we will also refer to this 
state as (max = state. The system is effectively frozen into one node in the history space. 
In this case, there are effectively only two kinds of strategies, which differ by their predictions 
for the history 111. The difference in predictions for the other (2™ — 1) history bit-strings 
become irrelevant. Obviously, the ranking in the performance of the two effective groups 
of strategies is such that the group of strategies that suggest an action '1' for the history 
'111', outperforms the group that suggests an action '0'. For a uniform initial distribution 
of strategies, there are A^/2* agents taking the action and (1 — 1/2*) agents taking the 
action '1', since half of the strategies predict and half of them predict 1. To sustain a 
winning outcome of '1', the criterion is that the resource level L should be higher than the 
number of agents taking the action '1'. Therefore, we have for the state with Wmax = 1 that 



These results are in agreement with the results obtained by numerical simulations. For 
s = 2, (w) = 3/4 for L > 3A^/4. Note that Equations (0) and Q are valid for any values of 
m. 

Table Hvl shows that the states with Wmax = 5/6, 6/7, 7/8 have very similar features in 
terms of the bit-string statistics. They differ only in the frequency of giving an outcome 
of 1 following the history of 111. Note that the C = 2 and C = 3 histories do not occur. 
The results imply that as the system evolves, the path in history space for these states is 
restricted to the two groups of histories labelled hj ( = and C = 1- The statistics show 
that the outcome bit-strings for the states with Wmax = 5/6, 6/7 and 7/8 exhibit only one 
0-bit in a period of 6, 7 and 8 bits, respectively. We refer to these states collectively as 
(max = 1 states, since the portion of allowed history space is bounded by the ( = 1 histories. 
Graphically, the path in history space consists of a few self-loops at the node 111, i.e. from 
111 to 111, then passing through the C = 1 group of histories once and back to 111, as 
shown in Fig. |2fb). The states with Wmax = 1/2, 12/17, 17/23 involve the other groups of 
histories and exhibit complicated looping among the histories. We refer to them collectively 




(1) 



and 




(2) 
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as higher (i.e. (max > 1) states. 
VI. THE Cmax = 1 STATES 

A. Values of Wmax 

We now proceed to derive an expression for the observed value of Wmax for the (^.ax = 
1 states. Recall that each strategy consists of a prediction or action for all of the m- 
bit histories. An important idea is that for states corresponding to paths restricted to 
a certain portion of the history space, only that part of a strategy corresponding to the 
histories in question is being used in making decisions. Strategies that only differ in their 
predictions for the history bit-strings which do not occur (i.e. the avoided histories) are now 
effectively identical. In the context of the Crowd- Anticrowd theory 0,0!^^! two previously 
uncorrelated strategies could now be correlated when viewed within this restricted history 
subspace. 

As L decreases, the system is allowed to explore a larger portion of the history space. The 
ranking in the strategies' performance becomes more complicated. For the (max = 1 states, 
the paths in history space involve the (m + 1) histories labelled by ( =0 and 1 (see Fig. |2fb)). 
For m = 3, only four out of a total of 2™ = 8 entries in a strategy corresponding to the 
histories '111', 'Oil', '110', and '101' now matter. For general m, there are m histories with 
one 'O'-bit: hence a total of (m+1) entries in each strategy now matter. For later discussions, 
it is useful to first classify all strategies into two groups according to their prediction for the 
history '11. . . 1' (C = history). These strategies can further be classified according to their 
m predictions for the C = 1 histories. A strategy having i bits predicting 1 and m — i bits 
predicting for the m histories belonging to C = 1? can be labelled as (/i; i, m — i) where 
/i = 0, 1 is the prediction for the C = history. The first three columns in Table |3 show 
this classification of strategies for general values of m describing the (max = 1 states. 

For L < 3A^/4, the outcomes are no longer persistently '1', and the system explores both 
the C = and ( = I groups of histories. Assume that for L just below 3A^/4, the outcome 
must be '1' for the C = 1 histories, i.e. the system only visits the C = 1 histories once in a 
cycle. Consider the m = 3 case, for example. As time evolves, the system exhibits periodic 
visits to the histories. In each period, each history in = 1 occurs once and the history in 
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C = occurs n + 1 times. Among these n + 1 occurrences of the C = history, the outcomes 
are '1' for n timesteps and '0' for one timestep (Fig. |2fb)), since the system must go from 
'111' to '110' after n loops in order to sustain the path. It turns out that paths in the history 
space for the Wmax = 5/6, 6/7, and 7/8 states (m = 3) correspond to that shown in Fig.|2fb) 
with n = 2, 3, and 4 loops at the C = history. 

Since only m + 1 different histories are involved, the performance of a strategy depends 
only on the predictions for this subset of histories. Consider the path in Fig. [2tb). The 
strategy labelled by (/i; i,m — i) predicts the correct outcome i + n times for /i = 1 and 
i + 1 times for /i = 0, respectively, in going through the path once. In Table I3 we list the 
performance of the strategies labelled by i,m — i) according to the number of successful 
predictions (which reflects the VPs) in a closed path (see Fig. I21b)) consisting of n loops 
at C = history, i.e. a total of (m + n + 1) timesteps from n = 1 to n = m + 2. It is 
important to note that there may be overlaps in strategies' performance between the /i = 1 
and /i = groups of strategies for small values of n, i.e. strategies with the label /i = 1 and 
/i = may win the same number of timesteps in a cycle and hence belong to the same rank 
in performance of the strategies. For example for m = 3 and n = 2, strategies labelled by 
(1; 2, 1) and (0; 3, 0) belong to the same rank in performance. 

The number of turns n around the C = history which is consistent with the condition 
of L < 3A^/4, is restricted to the range 2 < n < m + 1. This criteria is related to the 
number r of overlapping performances between the (1; z, m — z) and (0; j, m — j) groups of 
strategies (see Table IVjl . Note that for n > m + 1, we have r = 0, i.e. strategies with /i = 1 
do not have overlapping VPs with strategies with /i = 0. This implies that the VPs of the 
strategies predicting '1' for the history in = 0, are always higher than those predicting 
'0'. This further implies that agents will take action '1' for the C = history if one of their 
strategies belongs to the fi = 1 category. For s = 2, there will then be SN/A agents taking 
the action '1'. Since L < SN/i, the outcome must be '0'. Therefore, the upper bound for 
the number of self- loops is m + 1, and thus n < m + 1. In other words, for paths with 
n > m + 1 the system must have L > SN/A and the path in history space will be that 
of an infinite number of loops around C = history, i.e. the (max = state. For n < 1, 
strategies in (l;i,m — i) will perform worse than or equally well to those in (0;j, m — j) 
for i > j. As L > N/2, strategies that predict more I's should perform better. Therefore, 
n < 1 leads to inconsistency and thus n > 2. Thus 2 < n < m + 1 for the (max = 1 states. 
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with the corresponding r of overlapping groups of strategies being in the range 1 < r < m. 
Since each possible allowed value of or r gives one (max = 1 state, there are altogether m 
possible (max = 1 states for a given m. 

The values of Wmax for the (max = 1 states can be readily found. For a given value of n, 
the best performance among the strategies is to have m + n correct predictions in a path 
consisting of (m + n + 1) timesteps with n loops at the C = history. Therefore, for the 
(max — 1 states 

_ m + n 

U^max — , , , • V"jJ 

m + n -\- 1 

For m = 3, we have 2 < n < A and hence n = 2,3, 4. There are three (max = 1 states with 
Wmax = 5/6, 6/7, and 7/8, exactly as observed in the numerical simulations. 

B. Resource levels at transitions 

We now derive the critical values of the resource level at which transitions occur from 
one value of Wmax to another for the (max = 1 states. Note that the transition from the 
Wmax = 1 state to the Wmax = 7/8 state for m = 3 and s = 2, occurs at L = 3N/4 as 
predicted in Eq.(2). The condition for transitions from Wmax = 7/8 to Wmax = 6/7 state is 
that the value of L can no longer support n = 4 loops at the C = history before giving an 
outcome of '0' for the history 111. From Table \V\ we note that the performance of the /i = 1 
group of strategies becomes increasingly better than the /i = group as n increases, and 
the number of agents taking the action '1' increases towards 3N/4. Therefore, the highest 
number of agents who take the action '1' and win will arise at the last turn among the n 
loops where the history 111 is followed by an outcome '1'. Similarly, the lowest number of 
winning agents will be at the turn when the history 111 is followed by an outcome '0', i.e. 
breaking away from the n loops at 111. 

The number of agents choosing the action '1' given the history 111, is related to the 
number r (and hence n) of overlapping performances among the {l]i,m — i) and (0; j, m — j) 
strategies. From Table El the number of correct predictions Vf^{i) of a strategy (/x; i,m — i) 
in going through a path with n loops at the C = history, is given by 

= { (4) 

i + l ,/i = 0. 
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For general values of m, there are only (m + 1) histories which matter for the (max = 1 
states. Therefore there are 2'""'"^ effectively different strategies, each of which represents a 
group of 2^'"/2'^+^ strategies. The number of effectively different strategies predicting /i for 
the C = history and having i predictions of '1' for the m ( = 1 histories, is given by 

= cr (5) 

for both yU = 1 and 0, where is the binomial coefficient. 

The performance of the strategies can be ranked by a label r, with r = 1, . . . ^r^ax and 
r = 1 representing the best performing group of strategies. A general situation in which 
there are r overlapping performances between the (l;z,m — i) strategies and (0; j, m — j) 
strategies, is shown in Table IVIl For the (max = 1 states, the best performing strategies 
belong to the /i = 1 group and the worst performing ones belong to the /i = group, with 
r overlapping rankings in between where the allowed range of r is 1 < r < m. The ranking 
r of the strategies (/i; i,m — i) is related to i by the simple relation 

(6) 

2m + 2 — r — r ,/i = 0. 

For a given value of r, there are a total of 2m + 2 — r ranks. Therefore, the label r is 
restricted to the range 1 < r < Vmax with Vmax given by 

Tmax = 2m + 2 - T. (7) 

As the number of loops n increases, r decreases and the strategies spread more widely in 
terms of performance. It follows from Eqs. © and (jH} that the number of effectively different 
strategies c(r) in rank-r is given by 



c(r) 



CT-i ,rG(l,m + l-r) 

Cr-i + Cr_-. , r G (m + 2 - r, m + 1) (8) 
Cr__. , r e (m + 2,w). 



The fraction of rank-r strategies among all the strategies is then given by 

= |gr. (9) 

On the critical turn that determines the minimum value of L for sustaining a certain 
(max = 1 state, only the /x = 1 strategies win. Using Table |VT1 together with Eqs. (0) and 
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(jni), the number of winning strategies ci(r) is 



ci(r) 



(10) 



, r G (1, m + 1) 
,r e {m + 2,rmax), 

The corresponding fraction /i(r) of winning strategies among all strategies of rank-r is given 
by 

1 



ci(r) 
c(r) 



i+c;! 




,r e (1, m + 1 — r) 
, r € (m + 2 — r, m + 1) 
,r e (m + 2,rmax)- 



(11) 



Each agent uses the strategy in his possession which has the best performance record, i.e. 
the one having the ranking with smaller r, in order to make a decision. Assuming a uniform 
initial distribution of any combination of s strategies (with repetitions allowed) among the 
agents, the fraction of agents holding a rank-r strategy as their best performing strategy 



riHyr) is 



c r 



£ c(r') 



r'=r+l 



(12) 



with c(r) given by Eq.(P|). 

Each Cmax = 1 state corresponds to a specific value of allowed n and hence an allowed r. 
For a given n or r, the resource level L needed to accommodate all the agents that take the 
action '1' for the C = history, gives the criterion for the state: 



(13) 



r=l 



Note that rmax, fiij)i and nuir) are all r-dependent (see Eqs.Q, (fTT|) . and (fT^V Equation 
(jl3|) gives the lower bounds of L for each of the Cmax = 1 states. Note that the lower 
bound for a state with a given r is also the upper bound for the state with r + 1. For 
= 1001, s = 2, and r = 1, 2, and 3, Eq.lfT^ gives the lower bounds of L = 745, 695, and 
640 for the states characterized by Wmax = 7/8, 6/7, and 5/6, respectively. These values 
are in excellent agreement with those obtained by numerical simulations (see Fig. ^a) and 
Table HJ). We note that our approach of focusing on strategy-performance ranking patterns 
and the fraction of strategies in each rank, represents a generalization of a similar approach 
2^ that has already been successfully applied to the MG to cases in which some of the 
strategies have runaway VPs. 
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VII. DISCUSSION 



We have studied numerically and analytically the effects of a varying resource level L on 
the success rate of the agents in a competing population within the B-A-R model. We found 
that the system passes through different states, characterized either by the mean success 
rate (w) or by the highest success rate in the population Wmax, as L decreases from the 
high resource level limit. The number of states depends on details of the system such as 
the memory size m and the number of strategies per agent s. Transitions between these 
states occur at specific values of the resource level. For small values of m, it is possible 
to explain these states by following the evolution of the performance of the strategies and 
the decision making dynamics of the system. More generally, we found that different states 
correspond to different paths covering a subspace within the whole history space. In the 
high resource level regime, namely L > (1 — 1/2^)N, Wmax = 1- The corresponding path in 
the history space is one that loops around the history '111...' indefinitely. Just below the 
high resource level regime is a range of L that gives m states corresponding to the fractions 
Wmax — {m + n)/{'m + n + 1). This result is in excellent agreement with that obtained by 
numerical simulations. For these (^^ax — 1 states, i.e. the outcome series consists of one bit 
of '0' in a cycle of m+n+1 bits, the path in history space is restricted to those m-bit histories 
with at most one-bit of '0' and with n loops around the '111...' history. This identification 
of an active portion within the history space implies that only part of each strategy is being 
used. By considering the performance of the strategies within this restricted portion of the 
history space, the number of loops n consistent with the (^ax — 1 states was found to be 
2 < n < m + 1. The number of agents using a strategy that predicts the action '1' given 
the history '111...' increases as the number of loops n increases. Thus a criterion on the 
resource level for sustaining a state of given n can be derived. The results arc again in 
excellent agreement with numerical results. After passing through the (max = 1 states, the 
system goes into states with more than one 'O'-bit per cycle in the outcome series as L is 
further reduced. These (max > 1 states correspond to paths that explore an increasingly 
larger portion of the history space. While our analysis can also be applied to these (max > 1 
states, the dynamics and the results are too complicated to be included here. 

A resource level L that deviates from N/2 acts like a driving force in the history space. 
In response to this driving force, the system effectively adjusts its dynamics to occupy 
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an increasingly restricted portion of the history space as L increases. For L = N/2, the 
system explores the whole history space by passing through trails that are almost Eulerian 



24| . The random initial distribution of strategies and the random initial history bit-string 



that started the system, provide the seed for the diffusive behavior which develops in the 
history space as the system evolves. In fact, results of numerical simulations for L > 
N/2 show that slightly increasing L beyond N/2 has the effect of suppressing this random 
wandering through the history space, and instead locks the system into the Eulerian Trail. 
However opposing mechanisms can arise to counteract this driving force, thereby enhancing 
the diffusive behavior. For example, this can be achieved by allowing the agents the chance 



of using a strategy besides the best-performing one [31|, |3^ or by allowing some agents to 
opt out of the system occasionally |123|]. Alternatively, the system can be biased through the 
initial strategy scores, or by introducing a specially prepared non-random initial allocation 
of strategies j2J|. It is this competition between the diffusive and driven behavior that 
gives the non-trivial global behavior in the B-A-R model and its variations. For this reason, 
the present B-A-R system provides a fascinating laboratory for studying correlated, non- 
Markovian diffusion on a non-trivial network (i.e. history space). This non-trivial diffusion is 
in turn strongly coupled to the non-random temporal patterns arising in the VP dynamics. 
We note that the present results could also be used to generalize the Crowd-Anticrowd 
theory in order to incorporate the effect of restricted history-space dynamics: this would 
then allow identification of an appropriate set of correlated, uncorrelated and anti-correlated 
strategies in order to implement the Crowd-Anticrowd theoretical expressions. 

As a side-product, our analysis serves to illustrate the sensitivity within multi-agent 
models of competing populations, to tunable parameters. By tuning an external parameter, 
which we take as the resource level in the present work, the system is driven through differ- 
ent paths in the history space which can be regarded as a 'phase space' of the system. The 
feedback mechanism, which is built-in through the decision making process and the evalua- 
tion of the performance of the strategies, makes the system highly sensitive to the resource 
level in terms of which states the system decides to settle in or around. These features are 
quite generally found in a wide range of complex systems. The ideas in the analysis carried 
out in the present work, while specific to the B-A-R model used, are also applicable to other 
models of complex systems. 

In closing, we remark that besides obtaining analytically the highest success rate Wmax 
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and the criteria on the resource level L, our treatment can also be extended to obtain the 
mean success rate {w) for the (max = 1 states. The analysis is more complicated than that 
for obtaining Wmax- The procedure is to follow the evolution of the performance of the 
groups of strategies in each timestep through a path in the history space. The number of 
agents taking a particular action, and hence the number of winning agents, can be found 
from a strategies' performance table like the one shown in Table El We have carried out 
the analysis for (w) for m = 1, 2, and 3, and results are found to be in excellent agreement 
with numerical results. Our analysis can also be readily extended to consider connected 
populations in which agents have established links to connected neighbors for collecting 



additional information 
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29|. Quite generally, the effect of the links is to modify the 



number of agents using a strategy in a particular rank. For connected populations, an agent 
may use a strategy that he does not hold but has access to through his links. For an agent 
who uses the best performing strategy among his own s strategies and those of his connected 
neighbors, the success rate behaves in a similar fashion as a function of L as that reported 
here, only that the critical values of resource level at which transitions occur are shifted 



These results can be understood by incorporating the effects of the linkages into 
Results for the B-A-R model in a connected population will be reported elsewhere 
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'l'-bits:'0'-bits in a period 


Range of L 


Period 


1/2 


8:8 


- 510-600 


length 16 


12/17 


12:5 


~ 600-620 


length 17 


17/23 


17:6 


~ 620-640 


length 23 


5/6 


5:1 


- 640-695 


111110 


6/7 


6:1 


~ 695-745 


1111110 


7/8 


7:1 


~ 745-755 


11111110 


1 


1:0 


~ 755-1000 


1 



TABLE I: Table showing the states characterized by Wmax for B-A-R model with N = 1001 
agents, m = 3 and ,5 = 2, together with the ratio of number of occurrences of 'I'-bits to 'O'-bits 
in the outcome bit-string and the range of resource level in which the state occurs. The results 
are obtained from the numerical data as shown in Fig. ^a). The last column shows the period 
observed in the outcome bit-string. 
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Initial history = 



Timestep 




outcome 






(VP)(,j,) ni(t) 




=> (initial history) 


1 




(0 0) In 

\0 OJ 2^^ 
^ 1 




2 




f-i In 

\+l +1J 2-'^ 
^ 1 




o 
O 





(-2 OW ^ 


1 


A 


f-i -1) 3^ 

\+l +1J 4^^ 


C-3 +n 3^ 

V-1 +3/ 4^^ 







^ 


^ 1 


5 




(-2 0) Un 
VO -1-2/' 16-'^ 

^ 1 


f-^ +2) 3 
\-2 +4) 4^^ 


6 




f-S 5^ 
V-l-1 -1-3/ 8^^ 

=> 1 




7 

8 




^0 +4/ 16^^ 








16 -'^ 


fiV 




VP ranking pat- 
tern eventually 
repeats in four 
timcstcps, with 
a 4-bit period of 
1100 


VP ranking pat- 
tern eventually 
repeats in four 
timcstcps, with 
a 4-bit period of 
1110 


VP ranking pat- 
tern eventually 
repeats in every 
timestep, with 
the outcome be- 
inp persistently 



TABLE II: Time evolution of a B-A-R system for m = 1 and s = 2. The virtual points {yP){xy) of 
the strategies {xy) are given for a few timesteps, together with the number of agents ni{t) taking 
the action '1' and the outcome of each timestep in the format given in the first row of the table. 
Initially, the VPs of all strategies are set to zero and a uniform initial distribution of strategies in a 
large N population is assumed. The system settles into different states depending on the resource 
level L. 
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'l'-bits:'0'-bits in a period 


Range of L 


{w) 


2 

4 


2:2 


500-688 


25 
64 


3 
4 


3:1 


688-751 


9 

16 


1 


1:0 


751-1000 


3 
4 



TABLE III: Values of the mean success rates {w) for states corresponding to different resource 
level L in a B-A-R model of = 1001 agents, m = 1 and s = 2, obtained analytically by following 
the dynamics of the system as shown in Table Results are in excellent agreement with the 
simulation data given in Figure Q^b). 
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_ 8 
16 


^0 


^1 




12 
17 


^0 


^1 




17 
23 


^0 


^1 


C = 3 


000 


1 


1 




000 










000 










001 


1 


1 




001 





1 




001 





1 


C = 2 


010 


1 


1 




010 





1 




010 





1 




100 


1 


1 




100 





1 




100 





1 




on 


1 


1 




oil 


1 


2 




oil 


1 


3 


C = i 


101 


1 


1 




101 


1 


2 




101 


1 


3 




110 


1 


1 




110 


1 


2 




110 


1 


3 


C = o 


111 


1 


1 




111 


2 


3 




111 


3 


5 




8 8 




5 


12 




6 


17 



5 
6 


^0 


^1 


6 
7 


^0 


^1 


7 

8 


^0 


^1 


1 


^0 


^1 


000 








000 








000 








000 








001 








001 








001 








001 








010 








010 








010 








010 








100 








100 








100 








100 








oil 





1 


oil 





1 


on 





1 


oil 








101 





1 


101 





1 


101 





1 


101 








110 





1 


no 





1 


110 





1 


no 








111 


1 


2 


111 


1 


3 


111 


1 


4 


111 





1 




1 


5 




1 


6 




1 


7 







1 



TABLE IV: Outcome statistics of a B-A-R model with N = 1001 agents, m = 3 and s = 2 for states 
corresponding to different resource level L. The table shows the relative number of occurrence of 
each outcome following every possible history bit-string for the sta-tes charctcterized. by w^nax — 
8/16, 12/17, 17/23, 5/6, 6/7, 7/8, and 1. The parameter C, labels groups of histories as defined in 

Fig.m 
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history 


(=1 histories 


number of correct 
closed path with n 
tory 


predictions in a 
loops at ^=0 his- 






^J' 


^1 


^0 


1 


2 


n 


m + 1 


m + 2 


1 


11 i 


n 


TYl —1— 1 

III ± 


lit £j 


m + n 


2m + I 


2m + 2 


1 


m — 1 


1 


m 


m + 1 


m + n — 1 ■ ■ ■ 


2m 


2m + 1 


1 


i 


m — i 


i + l 


z + 2 


n + i 


m + i + 1 m + i + 2 


1 





TTi 


1 


2 


n 


m + 1 


m + 2 





m 





m + 1 


m + 1 


m + 1 


m + 1 


m + 1 





i 


m — i 


i + l 


i + l 


i + l 


i + l 


i + l 








m 


1 


1 


1 


1 


1 



TABLE V: All strategies can be labelled by (/i; i,m — i) as shown in the first three columns in the 
table. For paths with n loops as shown in Fig. E^b), the number of correct predictions by each 
group of strategies is given for different values of n from n = lton = m + 2. For the Cmax = 1 
states, 2 < n < m + 1. 
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r 


= 1 fi = 




1 
2 


'^max 
fmax 1 


m + 1 — r 
in = 1 strategies only) 


r 


'^max r + 1 Tjjig^x T -\- 1 


T 

(overlapping region) 




2 
1 


m + 1 — r 
(/X = strategies only) 



TABLE VI: Table showing the strategy performance ranking pattern, as reflected by the number 
of correct predictions f^(r), in the n-th loop in a path given in Fig. IHb) for Cmax = 1 states 
corresponding to turn with the largest winning crowd. Note that some = 1 groups of stategies 
may have overlapping cumulative performance with fi = groups of strategies. The total number 
of ranks rmax and the number of /i = and /U = 1 groups of strategies having identical rankings 
depend on n and m. 
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FIG. 1: The mean success rate {w) (dark solid line) as a function of the resource level L for a 
system with N = 1001 agents, s = 2 strategies per agent, with memory length (a) m = 3 and 
(b) m = 1. For each value of m, data for different values of L are taken in a system with the 
same initial distribution of strategies among the agents. Also shown are the range corresponding 
to one standard deviation about (w) in the success rates among the N agents (dotted lines) and 
the spread in the success rates given by the highest and the lowest success rates (thin solid lines) 
among the agents. 
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FIG. 2: (a) The history space in B-A-R model with m = 3. The nodes correspond to the 2"* 
possible histories. The transition between nodes are indicated by the arrows, together with the 
outcome needed for the transitions to occur. The histories can be grouped into columns labelled 
by a parameter ( which gives the number of '0' bits in the histories, (b) For the (max = 1 states at 
L < (1 — 1/2'*) A^, the system follows a path restricted to histories in the C = ^'^d ( = 1 columns, 
with n loops at the C = history. 
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